Image processing apparatus, method for processing image, and computer program

ABSTRACT

An image processing apparatus includes a taken image data acquiring unit configured to acquire taken image data, a storage unit configured to store therein one or more pieces of interface data for designating a tag to be attached to the taken image data, an imaging situation information acquiring unit configured to acquire information on an imaging situation of the taken image data, an interface identifying unit configured to identify at least one piece of interface data from among the pieces of interface data stored in the storage unit based on the information on the imaging situation acquired by the imaging situation information acquiring unit, and a display image generating unit configured to generate interface image data from the interface data identified by the interface identifying unit and to generate image data for display from the generated interface image data and the taken image data.

CROSS-REFERENCE TO RELATED APPLICATION(S)

This application is a continuation of International Application No.PCT/JP2013/053707, filed on Feb. 15, 2013 which claims the benefit ofpriority of the prior Japanese Patent Application No. 2012-044754, filedon Feb. 29, 2012, the entire contents of which are incorporated hereinby reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an image processing apparatus, a methodfor processing an image, and a computer program.

2. Description of the Related Art

A technology is known that when taking moving images and still imagesusing an imaging apparatus such as a video camera, a user automaticallyor manually gives to taken image data tags (information on a shootinglocation, a photographer, or a photographic subject, for example) foridentifying and searching for the taken image data. By using the tags,the user can easily perform organization and searching of the takenimage data.

Japanese Patent Application Laid-open No. 2010-21638, for example,discloses a technology that gives surrounding keywords derived from theposition of an imaging apparatus and map data to an image frame duringimaging as tags. Japanese Patent Application Laid-open No. 2007-228608discloses a digital still camera that displays a human figure icon rowand a genre-specific icon row related to tags on a display screen andcan easily give the tags to an image during imaging by selecting icons.

However, the imaging apparatus disclosed in Japanese Patent ApplicationLaid-open No. 2010-21638 and the digital still camera disclosed inJapanese Patent Application Laid-open No. 2007-228608 do not allow auser to easily select, in accordance with an imaging situation or thelike, a tag suitable for the imaging situation or the like from amongvarious tags and to give the tag to taken image data. It is consideredthat, for example, the type and contents of a tag a user desires to givediffer in accordance with the type of sports (baseball, soccer,basketball, or the like) to be imaged. However, the imaging apparatusdisclosed in Japanese Patent Application Laid-open No. 2010-21638 andthe digital still camera disclosed in Japanese Patent ApplicationLaid-open No. 2007-228608 cannot easily select and give a tagappropriate for the type of these sports from among various tags.

SUMMARY OF THE INVENTION

It is an object of the present invention to at least partially solve theproblems in the conventional technology.

The present invention provides an image processing apparatus thatincludes: a taken image data acquiring unit configured to acquire takenimage data; a storage unit configured to store therein one or morepieces of interface data for designating a tag to be attached to thetaken image data; an imaging situation information acquiring unitconfigured to acquire information on an imaging situation of the takenimage data; an interface identifying unit configured to identify atleast one piece of interface data from among the pieces of interfacedata stored in the storage unit based on the information on the imagingsituation acquired by the imaging situation information acquiring unit;and a display image generating unit configured to generate interfaceimage data from the interface data identified by the interfaceidentifying unit and to generate image data for display from thegenerated interface image data and the taken image data.

The present invention also provides a method for processing an imagethat includes: acquiring taken image data; acquiring information on animaging situation of the taken image data; referring to a storage unitthat stores therein one or more pieces of interface data for designatinga tag to be attached to the taken image data based on the acquiredinformation on the imaging situation and identifying at least one pieceof interface data; and generating interface image data from theidentified interface data and generating image data for display from thegenerated interface image data and the taken image data.

The present invention further provides a computer program that causes acomputer to execute: acquiring taken image data; acquiring informationon an imaging situation of the taken image data; referring to a storageunit that stores therein one or more pieces of interface data fordesignating a tag to be attached to the taken image data based on theacquired information on the imaging situation and identifying at leastone piece of interface data; and generating interface image data fromthe identified interface data and generating image data for display fromthe generated interface image data and the taken image data. The presentinvention also provides an image processing apparatus that includes: astorage unit configured to store therein one or more pieces of interfacedata for designating a tag to be attached to taken image data; animaging situation information acquiring unit configured to acquireinformation on an imaging situation of the taken image data; aninterface identifying unit configured to identify at least one piece ofinterface data from among the pieces of interface data stored in thestorage unit based on the information on the imaging situation acquiredby the imaging situation information acquiring unit; and

an output unit configured to output the interface data identified by theinterface identifying unit. The above and other objects, features,advantages and technical and industrial significance of this inventionwill be better understood by reading the following detailed descriptionof presently preferred embodiments of the invention, when considered inconnection with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram illustrating the appearance of an imaging apparatusaccording to a first embodiment;

FIG. 1B is a diagram illustrating the appearance of the imagingapparatus according to the first embodiment;

FIG. 2 is a block diagram illustrating the configuration of the imagingapparatus according to the first embodiment;

FIG. 3 is a block diagram illustrating the configuration of a centralcontroller according to the first embodiment;

FIG. 4A is a conceptual diagram illustrating a specific example of aninterface according to the first embodiment;

FIG. 4B is a conceptual diagram illustrating a specific example of theinterface according to the first embodiment;

FIG. 5 is a conceptual diagram illustrating the configuration ofposition-UI correspondence information according to the firstembodiment;

FIG. 6 is a conceptual diagram illustrating a specific example of animage for display according to the first embodiment;

FIG. 7 is a flowchart illustrating the operation of the imagingapparatus according to the first embodiment;

FIG. 8 is a block diagram illustrating the configuration of a centralcontroller according to a second embodiment;

FIG. 9A is a conceptual diagram illustrating a specific example ofgroup-UI correspondence information according to the second embodiment;

FIG. 9B is a conceptual diagram illustrating a specific example of thegroup-UI correspondence information according to the second embodiment;

FIG. 9C is a conceptual diagram illustrating a specific example of thegroup-UI correspondence information according to the second embodiment;

FIG. 10 is a flowchart illustrating the operation of an imagingapparatus according to the second embodiment; and

FIG. 11 is a conceptual diagram illustrating the configuration ofposition-UI correspondence information according to a third embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS First Embodiment

The following describes embodiments according to the present inventionwith reference to the drawings. FIG. 1A and FIG. 1B are appearancediagrams of an imaging apparatus 1 according to the first embodiment.The imaging apparatus 1 is a portable terminal apparatus such as asmartphone. FIG. 1A is a front view of the imaging apparatus 1, whereasFIG. 1B is a rear view of the imaging apparatus 1. The imaging apparatus1 has an imaging unit 100 for taking images on the rear side thereof andcan take still images and moving images. The imaging apparatus 1 alsohas on the front side a liquid crystal monitor 304 as a display unitthat displays the still images and moving images taken by the imagingunit 100. The imaging apparatus 1 according to the present embodiment isnot limited in its shape and constitution so long as it can take stillimages or moving images and includes a display unit (a liquid crystaldisplay, for example). In other words, the imaging apparatus 1 is notlimited to the portable terminal apparatus and may be any electronicapparatus (such as a digital video camera) having an imaging unit and adisplay unit.

Next, the configuration of the imaging apparatus 1 is described withreference to FIG. 1A, FIG. 1B, and FIG. 2. FIG. 2 is a block diagramillustrating the internal configuration of the imaging apparatus 1according to the present embodiment.

A central controller 400 includes a semiconductor integrated circuitincluding a central processing unit (CPU), a read only memory (ROM)storing various kinds of computer programs, and a random access memory(RAM) as a work area and comprehensively controls the processing of theentire imaging apparatus 1 such as imaging, display of various kinds ofimages, and identification of an image (hereinafter, referred to as an“interface image”) related to a graphical user interface (GUI) in whichicon images corresponding to the type or the like of tags used inaccordance with an imaging situation are appropriately arranged in orderfor a user to input the tags to taken image data. The internalconfiguration of the central controller 400 is described below withreference to FIG. 3.

The imaging apparatus 1 includes the imaging unit 100 including a zoomlens 101, a focus lens 102, an aperture 103, and an imaging element 104.The zoom lens 101 moves along an optical axis LA by a zoom actuator (notillustrated). Similarly, the focus lens 102 moves along the optical axisLA by a focus actuator (not illustrated). The aperture 103 operates bybeing driven by an aperture actuator (not illustrated). The imagingelement 104 includes a charge coupled device (CCD) and a complementarymetal oxide semiconductor (CMOS).

Imaging using the imaging unit 100 is performed as follows. The imagingelement 104 performs photoelectric conversion on light passed throughthe zoom lens 101, the focus lens 102, and the aperture 103 to generatean analog image signal of a subject. An analog image signal processingunit 105 amplifies this analog image signal, and then an image A/Dconverting unit 106 converts the amplified signal into digital imagedata. An image input controller 107 captures the digital image dataoutput from the image A/D converting unit 106 as taken image data andstores it in a main memory 206 via a bus 200.

A digital signal processing unit 108 captures the taken image datastored in the main memory 206 based on an instruction from the centralcontroller 400 via the bus 200 and performs certain signal processingthereon to generate data including a brightness signal and acolor-difference signal. The digital signal processing unit 108 alsoperforms various kinds of digital correction such as offset processing,white balance adjustment processing, gamma correction processing, RGBcomplementary processing, noise reduction processing, contour correctionprocessing, color tone correction processing, and light source typedetermination processing.

A microphone 109 collects ambient voices during imaging to generate ananalog voice signal. An analog voice signal processing unit 110amplifies the analog voice signal, and then a voice A/D converting unit111 converts the amplified signal into digital voice data. A voice inputcontroller 112 stores the digital voice data output from the voice A/Dconverting unit 111 in the main memory 206 together with the taken imagedata.

A multiplexer 113 multiplexes the taken image data and the digital voicedata stored in the main memory 206 to generate stream data.

A compression/expansion processing unit 201 performs certain compressionprocessing on the stream data stored in the main memory 206 inaccordance with an instruction from the central controller 400 via thebus 200 to generate compressed data. The compression/expansionprocessing unit 201 performs certain type of expansion processing on thecompressed data stored in the main memory 206 or the like in accordancewith an instruction from the central controller 400 to generateuncompressed data. The imaging apparatus 1 according to the presentembodiment employs a compression system compliant with JPEG standardsfor still images and a compression system compliant with MPEG2 standardsor AVC/H.264 standards for moving images.

A voice/image processing unit 202 performs certain image processing onthe digital data read from the main memory 206 in accordance with aninstruction from the central controller 400 via the bus 200. Thevoice/image processing unit 202 generates, for example, image data forvarious kinds of processing such as a menu image and an on screendisplay (OSD) image, superimposes the image data on the original takenimage data read from the main memory 206, and outputs the superimposedimage data to the liquid crystal monitor 304. This output causes theimage displayed on the liquid crystal monitor 304 to be an image inwhich various kinds of image data are combined or superimposed. Thevoice/image processing unit 202 superimposes or combines the interfaceimage on the taken image and displays the superimposed or combined imageon the liquid crystal monitor 304. The details are described below. Inplace of the liquid crystal monitor 304, other monitors such as anorganic electro-luminescence (EL) monitor may be used.

A ROM 203 is connected to the central controller 400 via the bus 200 andstores therein control programs executed by the central controller 400,various data required for the control, and the like. A flash ROM 204stores therein various kinds of setting information related to theoperation of the imaging apparatus 1 such as user setting information.

A VRAM 205 is used as a temporary storage area for image data fordisplay. The main memory 206 is used as a temporary storage area forvarious kinds of image data and is also used as a computing work areafor the central controller 400.

A gyro sensor 211 detects changes in triaxial acceleration and angularvelocity. In accordance with a detection result of the gyro sensor 211,the display orientation of the liquid crystal monitor 304 is changed. Aclock 212 generates information on the acquisition date and time of thetaken image data, and the generated information may be used as a tag.

The liquid crystal monitor 304, a speaker 305, an operating unit 306,and an input/output terminal 307 are connected to an input/output I/F303. The liquid crystal monitor 304 displays, for example, imagesgenerated from various kinds of image data such as taken image datatemporarily recorded in the VRAM 205 or the main memory 206, interfaceimage data, and various kinds of menu data. It goes without saying thatthe taken image data output to the liquid crystal monitor 304 includesnot only image data recorded in a recording medium (not illustrated)such as a card type recording medium by turning on a record button (notillustrated) of the operating unit 306 by a user, but also data relatedto what is called a through image that is displayed on the liquidcrystal monitor 304 for the user to check contents to be imaged. Thespeaker 305 outputs, for example, a voice temporarily stored in the mainmemory 206. The operating unit 306 includes operating buttons or a touchpanel arranged on the liquid crystal monitor 304 and receives operationinput to the imaging apparatus 1 by the user. The input/output terminal307 is connected to a television monitor, a personal computer (PC) (notillustrated), or the like.

A wireless module 309 transmits and receives data to and from otherapparatuses via the bus 200 and a wireless I/F 308. Wirelesscommunication of various kinds of information, taken image data, or thelike is performed by a method compliant with IEEE802.11 standards. Morespecifically, the wireless module 309 performs communication processingcompliant with wireless LAN standards such as Wi-Fi.

A communication module 311 performs e-mail transmission and receptionwith other communication terminals and Internet connection via the bus200 and a communication I/F 310. Such wireless communication is, forexample, performed by a method compliant with communication standardssuch as W-CDMA and CDMA2000. This enables communication with a basestation. The communication standards of the communication by thecommunication module 311 are not limiting. The communication module 311performs wireless communication by a communication method different fromthe wireless module 309.

The following describes the internal configuration of the centralcontroller 400 and related configuration with reference to FIG. 3. FIG.3 appropriately omits the bus 200, the input/output I/F 303, and thelike. The central controller 400 includes an imaging situationinformation acquiring unit 401, an interface identifying unit 403, and adisplay image generating unit 404.

The imaging situation information acquiring unit 401 acquiresinformation on an imaging situation of the imaging apparatus 1. Theimaging situation is a concept including an imaging position, an imagingtime, an imaging person, a subject to be imaged (subject) and itsnumber, weather during imaging, and an analysis state (imaging a subjectwith large movement, for example) of an image (a still image or a movingimage) during imaging. The present embodiment describes a case as anexample in which the imaging situation information acquiring unit 401acquires information on the imaging position as the imaging situation.

The imaging situation information acquiring unit 401 includes a positioninformation acquiring unit 402. The position information acquiring unit402 acquires a current position where the imaging apparatus 1 ispresent. The position information acquiring unit 402 is, for example, aprocessing unit having a global positioning system (GPS) function. Theposition information acquiring unit 402, when starting imaging or thelike, acquires information on the current position and supplies theacquired current position to the interface identifying unit 403.

The main memory 206 stores therein a plurality of pieces of information(hereinafter, referred to as “interface data”) on interfaces forinputting tags in advance, and the interface identifying unit 403selects and identifies interface data suitable for the information onthe imaging situation acquired by the imaging situation informationacquiring unit 401 from among the plurality of the pieces of interfacedata stored in advance. When the number of the pieces of interface datastored in the main memory 206 in advance is one, if the interface datafits the information on the imaging situation acquired by the imagingsituation information acquiring unit 401, the stored interface data isidentified.

The display image generating unit 404 generates interface image datarelated to the identified interface data. The interface image isdisplayed on the liquid crystal monitor 304 together with the takenimage data.

The tag is information for use in identification or searching of images.Examples of the tag for moving images related to baseball may includeinformation on a venue, score information, opponent information, a dateand time, or the name of a meet.

The imaging apparatus 1 may use the type and contents of tags created inadvance or may use the type and contents of tags created by a userseparately. With regard to tags related to baseball, for example,information used for baseball in common including declarations byumpires such as “strike,” “ball,” “out,” or “change,” innings such as“top and bottom of the Nth inning,” or scoring form such as “added byone run” is used as the tags created in advance. The name of a meet, thenames of both teams, and the like are used as the tags created by a userseparately. Similarly, as tags related to basketball, for example,information used for basketball in common including declarations byreferees such as “foul” and “violation,” periods such as “the first tothe fourth,” or scoring format such as “added by one to three points” isused as the tags created in advance. The name of a meet, the names ofboth teams, and the like are used as the tags created by a userseparately.

The type and contents of the tags may be, for example, stored in themain memory 206 in advance or may be acquired from any storage device(what is called a USB memory or the like) that is attachable anddetachable to and from a server (not illustrated) or the imagingapparatus 1. The respective processing units within the centralcontroller 400 appropriately acquire the type and contents of the tagsas needed. The user of the imaging apparatus 1 appropriately inputstheses tags while taking moving images. After the end of the imaging,the taken image data is identified or searched for using the given tags,thereby enabling to organize the taken image data and to reproduce oredit desired scenes. The tags can also be given to still images.

The interface data includes information identifying related tags amongthe stored tags and is stored in the main memory 206.

The interface identifying unit 403 reads UI management information 207,position-UI correspondence information 208, and map information 209 thatare generated and stored in advance from the main memory 206. The UImanagement information 207 is management information including anidentifier and usage (“baseball,” “soccer,” and “basketball,” forexample) associated with each other for each interface data for one ormore stored interface data. FIG. 4A and FIG. 4B illustrate examples ofthe interface data, the interface image related to the interface data,and the UI management information.

FIG. 4A is an interface image (identifier=“Basketball UI”) generatedbased on the interface data for inputting tags while imaging basketball.As illustrated, the interface image is configured so that information onthe names of the teams, a scoring process, and the like can be input asthe tags. Similarly, FIG. 4B is an interface image (identifier=“SoccerUI”) generated based on the interface data for inputting tags whileimaging soccer.

In order to give tags of different types and contents to the same sport,a plurality of pieces of interface data may be prepared. Interface datafor match and interface data for practice (interface data capable ofinputting a practice menu as a tag, for example) may be configureddifferently, for example. It goes without saying that, without beinglimited to sports, interface data for imaging dramas, music concerts, orthe like may be present.

The position-UI correspondence information 208 is information indicatinga correspondence relation between position information and UI managementinformation associated with the interface data related to the interfaceimage displayed on the liquid crystal monitor 304. FIG. 5 is a diagramillustrating the configuration of the position-UI correspondenceinformation 208. The example of FIG. 5 defines that an interface with anidentifier of “Baseball UI” is used at Ballpark A (35.XX latitude and139.XX longitude) and that an interface with an identifier of“Basketball UI” is used at Gymnasium B (36.YY latitude and 139.YYlongitude).

The user may manually set the correspondence relation within theposition-UI correspondence information 208, and the correspondencerelation may be information downloaded from a network. Alternatively, itmay be configured that interface data selected manually by the user froman interface selection screen and position information when theselection operation is performed are successively reflected in theposition-UI correspondence information 208. For the ease of descriptionand understanding, it is assumed that in the present embodiment, theposition in the position-UI correspondence information 208 and theinterface data are associated with each other on a one-to-one basis.

The map information 209 is information indicating the relation betweenthe position information and a facility (a sports facility, for example)present at the position. The map information 209 is not required to bestored in the main memory 206 and may be downloaded from any server on anetwork.

The interface identifying unit 403 selects and identifies interface datato be displayed based on the information on the current position.Specifically, the interface identifying unit 403 searches whetherinterface data corresponding to the current position is set based on theUI management information 207 and the position-UI correspondenceinformation 208. When the corresponding interface data is present, theinterface identifying unit 403 identifies the interface data as anobject to be notified to the display image generating unit 404. Whenthere is no corresponding interface data, the interface identifying unit403 searches the map information 209 for any facility corresponding tothe current position. When there is a corresponding facility, theinterface identifying unit 403 identifies interface data to be displayedfrom the usage of the facility. When the current position is found to bea soccer stadium through the searching of the map information 209, theinterface identifying unit 403 notifies the display image generatingunit 404 of interface data whose usage is soccer and that is associatedwith an identifier of soccer. When there is no corresponding facility orwhen interface data cannot be uniquely determined (to be one), theinterface identifying unit 403 notifies the display image generatingunit 404 of an invalid value as the interface data (invalid value).

In the comparison processing with the current position, the interfaceidentifying unit 403 may determine that positions for which a certainlevel of physical closeness is found correspond to each other. Positionsseparated by a distance similar to the distance between a main stand anda rear stand of the same soccer stadium, for example, may be determinedto be the same position.

The taken image data (a still image or a moving image) and the interfacedata from the interface identifying unit 403 are input to the displayimage generating unit 404. The display image generating unit 404generates the interface image data including icon image data related toicon images to be displayed and data related to the arrangement of theicon images to be displayed based on the notified interface data. Thedisplay image generating unit 404 generates image data for display inwhich the taken image data and the interface image data are combined.

When the invalid value as the interface data is notified, the displayimage generating unit 404 displays only the taken image data on theliquid crystal monitor 304. Alternatively, the display image generatingunit 404 displays a screen prompting the user to select an interface onthe liquid crystal monitor 304. When the user selects an interface, theinformation on the current position may be acquired from the imagingsituation information acquiring unit 401, and a combination of thecurrent position and the UI management information on the selectedinterface data may be stored as new data of the position-UIcorrespondence information 208.

The image data for display generated by the display image generatingunit 404 is output from the input/output I/F 303 to the liquid crystalmonitor 304. The liquid crystal monitor 304 displays an image fordisplay related to the supplied image data for display. FIG. 6 is adiagram illustrating an example of the image for display displayed onthe liquid crystal monitor 304. As illustrated in FIG. 6, the liquidcrystal monitor 304 displays an image for display in which a taken imagebeing currently taken and a related interface image are combined. Theuser, while viewing the taken image displayed on a taken image area,appropriately gives tags to the taken image data using the interfaceimage displayed on a tag input interface area. When the user touches ahit button 501, for example, its time information and the tag “hit” areassociated with each other and are stored in the main memory 206 (or anystorage device). The same applies to an out button 502, a walk button503, and a change button 504. Similarly, when a run is scored, runbuttons 505 to 508 are touched (when a grand-slam homerun is hit, forexample, the run button 508 indicating four runs is touched), therebystoring the time and scoring process in the main memory 206 (or anystorage device). The interface image is not required to be displayed onthe lower part of the display screen of the liquid crystal monitor 304and can be arranged at any position and with any size. Other images suchas a menu image may be displayed in combination.

Described next with reference to the flowchart illustrated in FIG. 7 isthe setting operation of the interface image for inputting tags to thetaken image in accordance with the imaging situation by the imagingapparatus 1 according to the present embodiment. First, at the timing ofstarting imaging or the like, the position information acquiring unit402 acquires the information on the current position and supplies theinformation on the current position to the interface identifying unit403 as information related to the imaging situation (S11).

The interface identifying unit 403 identifies interface data related tothe interface image to be displayed based on the information on thecurrent position, the position-UI correspondence information 208, andthe map information 209 and notifies the display image generating unit404 of the identified interface data (S12). When the interface cannot beuniquely identified, the interface identifying unit 403 notifies thedisplay image generating unit 404 of the invalid value as the interfacedata (S12).

The display image generating unit 404 determines whether the interfacedata is the invalid value (S13). When it is not the invalid value (No atS13), the display image generating unit 404 generates the image data fordisplay in which the interface image data corresponding to the interfacedata and the taken image data are combined and supplies the image datafor display to the liquid crystal monitor 304 (S14). The liquid crystalmonitor 304 displays the image data for display. When the notifiedinterface data is the invalid value (Yes at S13), the display imagegenerating unit 404 supplies only the taken image data to the liquidcrystal monitor 304. The liquid crystal monitor 304 displays the takenimage data as it is (S15). At S15, the display image generating unit 404may generate interface selection image data to display a section screenrelated to the generated selection image data on the liquid crystalmonitor 304.

The imaging apparatus 1 according to the present embodimentautomatically selects the interface in accordance with the imagingsituation and displays the image for display in which the taken imageand the interface image generated based on the selected interface dataare combined on the liquid crystal monitor 304. In other words, evenwhen the user performs imaging operation by changing the imagingsituation, imaging and tag input processing can be performed using theimage for display in which an appropriate interface is displayed withoutselecting the interface by the user. The interface is automaticallyselected in accordance with the imaging situation, thereby enabling theuser to save the trouble of selecting the interface.

The present embodiment acquires the information on the imaging positionas the imaging situation and selects the interface in accordance withthe position. In general, it is often the case that once the imagingposition is determined, the subject to be imaged is also determined. Theinterface identifying unit 403 refers to the position-UI correspondenceinformation 208, which defines the relation between the position and theinterface, and selects the most suitable interface for the currentposition, thereby enabling to select the interface with high precision.

Even when the interface corresponding to the current position is notdefined in the position-UI correspondence information 208, the interfaceidentifying unit 403 can select the interface corresponding to thecurrent position by referring to the map information 209. Owing to this,even when imaging is performed at a location (a remote sports stadium,for example) where the user does not usually perform imaging, anappropriate interface can be selected.

For the ease of description and understanding, the above embodimentdescribes a case in which the position information in the position-UIcorrespondence information 208 and the interface data are associatedwith each other on a one-to-one basis and the interface identifying unit403 identifies one piece of interface data. The interface identifyingunit 403 can also identify a plurality of pieces of interface data fromone piece of position information.

In this case, a table is recorded that associates the positioninformation in the position-UI correspondence information 208 and theinterface data with each other on a one-to-many basis, and the pieces ofidentified interface data are subjected to processing such as allowingthe user to finally select the interface data to be used based on anorder of priority set in advance and displaying a plurality of interfaceimages related to interface data identified by, for example, reducingthe display size of the interface image data to be output.

Second Embodiment

Next, the second embodiment according to the present invention isdescribed. The present embodiment is characterized in that anappropriate interface is selected in accordance with a subject to beimaged (subject) or a group constitution of subjects to be imaged.

Although the configuration of the imaging apparatus according to thepresent embodiment is similar to the configuration of the firstembodiment illustrated in FIG. 2, the second embodiment is differentfrom the first embodiment in that information on a person or a group ofpersons to be imaged (hereinafter, simply referred to as a “group to beimaged;” the group to be imaged may include one person) is used as theinformation on the imaging situation. FIG. 8 is a block diagramillustrating the configuration (and its neighboring configuration) ofthe central controller 400 according to the present embodiment.

The imaging situation information acquiring unit 401 includes a groupinformation acquiring unit 405. The main memory 206 stores therein theUI management information 207 and group-UI correspondence information210.

The group information acquiring unit 405 acquires information(hereinafter, simply referred to as “subject information”) on a group ofsubjects to be imaged by the following method, for example. The groupinformation acquiring unit 405 supplies the acquired subject informationto the interface identifying unit 403.

The imaging apparatuses 1 held by respective users, for example,appropriately notify a server (not illustrated) that “imaging is startedat Location X,” “A-san and B-san are going to be imaged now,” or thelike. The server collects the information transmitted from therespective imaging apparatuses 1 and transmits information on a group tobe imaged from this time on to the respective imaging apparatuses 1. Thegroup information acquiring unit 405 acquires the subject informationbased on this information. When A-san, B-san, and C-san individuallynotify the server that “imaging is started at Location X” from therespective imaging apparatuses 1, for example, the server notifies therespective imaging apparatuses 1 that a group to be imaged of “A-san,B-san, and C-san” starts imaging. When A-san, B-san, and C-sanindividually notify the server that “imaging is started at Athletic MeetY,” the server notifies the respective imaging apparatuses 1 that agroup to be imaged of “A-san, B-san, and C-san” starts imaging.

Alternatively, the user can display a general address book interfaceimage on the liquid crystal monitor 304 and appropriately select a userto be imaged. The group information acquiring unit 405 acquires the userselected through operation on this address book interface image as thesubject information. The above method for identifying the group to beimaged is merely an example, and the group to be imaged may beidentified to acquire the subject information by using any other method.

Next, the group-UI correspondence information 210 is described. Thegroup-UI correspondence information 210 is person correspondenceinformation indicating the correspondence relation between the person orthe group of persons to be imaged and the UI management informationassociated with the interface data related to the interface image to bedisplayed. FIG. 9A is a diagram illustrating an example of the group-UIcorrespondence information 210. FIG. 9A defines that when “A-san andB-san” are the group of persons to be imaged, “Tennis UI” is used, andwhen “B-san” is the group of persons to be imaged, “Bike UI” is used, orthe like.

The group-UI correspondence information 210 may define detailedcorrespondence relations about groups. FIG. 9B illustrates an example.FIG. 9B defines that “when the subject to be imaged is a group includingA-san and B-san, Tennis UI is used,” “when the subject to be imaged isonly B-san, Bike UI is used,” “when the subject to be imaged partiallyincludes C-san, D-san, E-san, F-san, G-san, and H-san and includes lessthan 10 in all, Futsal UI is used,” “when the subject to be imagedpartially includes C-san, D-san, E-san, F-san, G-san, and H-san andincludes 10 or more in all, Soccer UI is used,” or the like. Thegroup-UI correspondence information 210 may be configured so that theuser can appropriately create or edit it.

With reference to FIG. 8 again, the operation of the interfaceidentifying unit 403 is described. The subject information is input fromthe group information acquiring unit 405 to the interface identifyingunit 403. The interface identifying unit 403 reads the group-UIcorrespondence information 210 from the main memory 206. The interfaceidentifying unit 403 identifies the interface data in accordance with acomparison between the subject information and the group-UIcorrespondence information 210. The interface identifying unit 403notifies the display image generating unit 404 of the selected interfacedata. When the group to be imaged and the corresponding interface dataare not defined in the group-UI correspondence information 210, theinterface identifying unit 403 notifies the display image generatingunit 404 of the invalid value as the interface data.

Described next with reference to the flowchart illustrated in FIG. 10 isthe setting operation of the interface image for inputting tags to thetaken image in accordance with the imaging situation by the imagingapparatus 1 according to the present embodiment. First, the groupinformation acquiring unit 405 acquires the subject information andsupplies the acquired subject information to the interface identifyingunit 403 (S21).

The interface identifying unit 403 identifies the interface data to bedisplayed based on the subject information, the UI managementinformation 207, and the group-UI correspondence information 210 andnotifies the display image generating unit 404 of the identifiedinterface data (S22). When the interface data cannot be uniquelyidentified, the interface identifying unit 403 notifies the displayimage generating unit 404 of the invalid value as the interface data(S22).

The subsequent pieces of processing (S13 to S15) are nearly the same asS13 to S15 of FIG. 7, and the detailed description thereof is omitted.Although the above example describes a case of using only theinformation on the group to be imaged as the information on the imagingsituation, the above position information may be used in combination. Inthis case, when the interface data cannot be uniquely identified by theprocessing at S22, the interface identifying unit 403 may identify theinterface image data based on the current position, the UI managementinformation 207, the position-UI correspondence information 208, and themap information 209.

When the position information acquiring unit 402 is present within theimaging situation information acquiring unit 401, association with boththe position information and the subject information may be takenconsideration in the group-UI correspondence information 210. Asillustrated in FIG. 9C, for example, correspondence relations with boththe position information and the subject information may be defined suchas “when the subject to be imaged partially includes I-san, J-san,K-san, L-san, M-san, and N-san and imaging is performed at Location C,Futsal UI is used” and “when the subject to be imaged partially includesI-san, J-san, K-san, L-san, M-san, and N-san and imaging is performed atLocation D, Beach Soccer UI is used.”

After the end of the imaging processing, the imaging apparatus 1 mayautomatically transmit tag-given taken image data attached to an e-mailto the imaging apparatuses (or image display apparatuses) of therespective users included in the group to be imaged. This enablestag-input images to be shared.

The present embodiment considers members of the group to be imaged andselects an interface appropriate for the members. This can automaticallyselect the most suitable interface for the activities of a given group.As illustrated in FIG. 9B, the correspondence relation between the groupto be imaged and the interface can be set by taking consideration ofpartial match (“partially include C to H-san,” for example) of groupmembers. This can automatically select an appropriate interface evenwhen enjoying sports with any newly invited member added to usualmembers.

For the ease of description and understanding, the above embodimentdescribes a case in which the subject information in the group-UIcorrespondence information 210 and the interface data are associatedwith each other on a one-to-one basis and the interface identifying unit403 identifies one piece of interface data. The interface identifyingunit 403 can also identify a plurality of pieces of interface data fromone piece of subject information.

In this case, a table is recorded that associates the subjectinformation in the group-UI correspondence information 210 and theinterface data with each other on a one-to-many basis, and the pieces ofidentified interface data are subjected to processing such as allowingthe user to finally select the interface data to be used based on anorder of priority set in advance or displaying a plurality of interfaceimages related to interface data identified by, for example, reducingthe display size of the interface image data to be output.

Third Embodiment

Next, the third embodiment according to the present invention isdescribed. The imaging apparatus 1 according to the third embodiment ischaracterized in that the most suitable interface is selected based onan interface use history. The configuration of the imaging apparatus 1according to the present embodiment is nearly the same as theconfiguration illustrated in FIG. 2 and FIG. 3, and the description ofthe duplicated parts is appropriately omitted.

FIG. 11 is a diagram illustrating the configuration of the position-UIcorrespondence information 208 according to the present embodiment. Asdescribed above, the position-UI correspondence information 208 is theinformation indicating the correspondence relation between the positioninformation and the interface data. In the present embodiment, theposition-UI correspondence information 208 includes the interface usehistory as the correspondence relation. In this case, the position-UIcorrespondence information 208 registers therein imaging dates and timestogether. Specifically, as illustrated in FIG. 11, the position-UIcorrespondence information 208 is history information including datarows that associate respective imaging locations, interfaces used at therespective imaging locations, and respective imaging dates and timeswith one another.

Every time the user performs imaging using a tag input interface, a newdata row is added to the position-UI correspondence information 208.When “Baseball UI” was used for imaging at “35.XX latitude and 139.XXlongitude” (hereinafter, also referred to as Location A) from 18 o'clockon Sunday, Apr. 9, 2011, for example, its use history is added as thedata row. Given this situation, there is a case (a case of one-to-manycorrespondence) in which different interfaces (“Baseball UI”, “Baseballpractice UI,” or the like) are associated with the same location(Location A, for example).

The interface identifying unit 403 determines whether data rowcorresponding to the current position is present within the position-UIcorrespondence information 208. When there is no data row correspondingto the current position, the interface identifying unit 403 notifies thedisplay image generating unit 404 of the invalid data as the interfacedata in the same manner as the first embodiment. When there is only onedata row corresponding to the current position, the interfaceidentifying unit 403 notifies the display image generating unit 404 ofthe interface data of the data row in the same manner as the firstembodiment.

When there are a plurality of data rows corresponding to the currentposition, the interface identifying unit 403 analyzes the plurality ofdata rows and uniquely identifies the interface data to be notified tothe display image generating unit 404 in accordance with an analysisresult. The following describes data analysis examples.

As a first example, the interface identifying unit 403 may identifyinterface data whose use date and time is the newest. In the example ofFIG. 11, the interface identifying unit 403 identifies the data relatedto “Baseball UI” as the interface data corresponding to Location A(because its use on Sunday, Apr. 9, 2011 is the newest).

As a second example, the interface identifying unit 403 may identifyinterface data that is most frequently used with respect to the currentposition. In the example of FIG. 11, the interface identifying unit 403identifies the data related to “Baseball UI” as the interface datacorresponding to Location A.

The interface identifying unit 403 may detect regularity of date andtime (date, day of the week, time zone, or the like) and identify theinterface data in accordance with the detected regularity. The interfaceidentifying unit 403, for example, analyzes an interface use pattern atthe current position. In the example of FIG. 11, at Location A,“Baseball practice UI” is used on weekdays (Monday through Friday),“Baseball practice match UI” is used on Saturday, and “Baseball UI” isused on Sunday. The interface identifying unit 403 compares the analysisresult taken into consideration of date and time (date, day of the week,time zone, or the like) with the current date and time and identifiesthe most suitable interface data in accordance with the comparisonresult. If the date and time when imaging is started is Sunday, forexample, the interface identifying unit 403 selects the data related to“Baseball UI” as the interface data corresponding to Location A.

The processing flow is similar to the flowchart illustrated in FIG. 7,and the detailed description thereof is omitted (the details of theselection processing of the interface identifying unit 403 at Step S12illustrated in FIG. 7 are different).

The present embodiment refers to the previous interface use history andselects an interface based on the analysis result of the use history.Interface selection based on the actual user's use pattern can beperformed, and the interface the user desires to use can be selectedwith high precision.

In particular, the user often does regular activities in accordance withdate and time (date, day of the week, time zone, or the like). Thepresent embodiment analyzes whether the user performs regular imagingbased on date and time (date, day of the week, time zone, or the like),thereby enabling to perform interface selection depending on theactivities of the user with high precision.

Other Embodiments

The imaging situation information acquiring unit 401 analyzes an imageduring imaging, and in accordance with the analysis result, theinterface identifying unit 403 may identify the interface data. Thefollowing describes a specific example.

Taken image data (a moving image) is supplied to the imaging situationinformation acquiring unit 401 from the main memory 206. The imagingsituation information acquiring unit 401 may include, in addition to theposition information acquiring unit 402 and the like, an image analyzingunit that analyzes a subject to be imaged, the amount of the movement ofthe subject to be imaged, a voice, or the like within the suppliedmoving image. The image analyzing unit, for example, reads images of atennis racket or a tennis ball stored in the main memory 206 anddetermines whether any matched object is present (that is, whether atennis racket or a tennis ball is present) within the moving image. Whenthe tennis ball or the tennis racket is detected, the interfaceidentifying unit 403 may identify the interface data related to tennis.

Similarly, the image analyzing unit analyzes whether any word (forsoccer, “offside,” “centering,” “sliding,” “volley shot,” or the like)related to a specific sport (or a drama) has been detected within thevoice of the moving images. In accordance with the analysis resultindicating whether any related word has been detected, the interfaceidentifying unit 403 may identify the interface data.

The image analyzing unit detects the amount of the movement of thesubject to be imaged within the moving image and determines whether theamount of the movement is large. The interface identifying unit 403 mayidentify interface data related to a sport frequently used when a movingimage in which the amount of the movement of the subject to be imaged islarge is taken and may identify interface data most frequently usedother than sports when a moving image in which the amount of themovement of the subject to be imaged is small is taken.

According to the configuration including the image analyzing unit, anappropriate interface depending on the subject to be imaged can beautomatically selected even when the interface cannot be identified bythe current position and the group to be imaged.

The analysis result of the moving image, the current position, and thelike are an example of the imaging situation, and the interface may beselected based on any other information (weather, for example) relatedto imaging. It goes without saying that the image analyzing unit mayperform the analysis of still images as well as the analysis of movingimages.

Although the present invention has been described with reference to theabove embodiments, it is understood that the present invention is notlimited to the constitutions of the above embodiments and includesvarious alterations, modifications, and combinations that can beachieved by those skilled in the art within the scope of the presentinvention.

It is understood that, for example, part of the information stored inthe main memory 206 may be stored in a storage device (a USB memory, forexample) that is attachable and detachable to and from the imagingapparatus 1. The imaging apparatus 1 may appropriately downloadinformation to be stored in the main memory 206 from a network. At leastpart of the information stored in the main memory 206 may be stored in astorage unit (not illustrated) within the central controller 400.

Any processing of the central controller 400 can also be achieved bycausing a central processing unit (CPU) to execute a computer program.In this case, the computer program can be stored using various types ofnon-transitory computer readable media to be supplied to a computer. Thenon-transitory computer readable media include various types of tangiblestorage media. Examples of the non-transitory computer readable mediamay include magnetic recording media (flexible disks, magnetic tapes,and hard disk drives, for example), magneto-optical recording media(magneto-optical disks, for example), compact disc read only memories(CD-ROMs), CD-Rs, CD-R/Ws, and semiconductor memories (mask ROMs,programmable ROMs (PROMs), erasable PROMs (EPROMs), flash ROMs, andrandom access memories (RAMs), for example). The computer program may besupplied to the computer by various types of transitory computerreadable media. Examples of the transitory computer readable media mayinclude electric signals, optical signals, and electromagnetic waves.The transitory computer readable media can supply the computer programto the computer via a wired communication path such as an electric wireand an optical fiber or a wireless communication path.

Embodiments of the present invention include not only a case ofachieving the functions of the above embodiments by causing the computerto execute a computer program that achieves the functions of the aboveembodiments but also a case of achieving the functions of the aboveembodiments by causing the computer program to cooperate with anoperating system (OS) or application software operating on the computer.In addition, embodiments of the present invention include a case ofachieving the functions of the above embodiments by performing the wholeor part of the processing of this computer program by a functionexpansion board inserted into the computer or a function expansion unitconnected to the computer.

Although the present invention is described above for the imagingapparatus, the present invention can also be applied to an apparatushaving no imaging capability. The following describes an applicationsystem including an apparatus having no imaging capability.

The application system includes an imaging apparatus being able to takea still image and a moving image (hereinafter, assumed to be a videocamera) and any communication apparatus (hereinafter, assumed to be aportable terminal apparatus) communicable with the imaging apparatus.The application system can be used for a case, for example, in which auser, while viewing a sport match at a match site, performs imaging withthe video camera using a pan head and appropriately inputs tags to takenimage data. The user automatically performs the imaging with the panhead set and operates the portable terminal apparatus, which is easy tobe held, to perform tag inputting operation while grasping the progressof the match with the naked eyes.

Although the portable terminal apparatus includes the interfaceidentifying unit 403 within the central controller 400 and the liquidcrystal monitor 304, it does not require to have imaging capability. Itis assumed that the portable terminal apparatus and the video camera arephysically closely positioned. Given this situation, even when theimaging situation information acquiring unit 401 is present within asmartphone and detects the above imaging situation, the detectedsituation can be regarded to be equal to the imaging situation of thevideo camera. The imaging situation information acquiring unit 401 maybe present only within the video camera, and the portable terminalapparatus may appropriately acquire information on the imaging situationfrom the imaging situation information acquiring unit 401 within thevideo camera.

The interface identifying unit 403 within the portable terminalapparatus selects the interface by the above method in accordance withthe acquired imaging situation and displays the selected interface imageon the liquid crystal monitor 304. The user appropriately inputs tagsthrough operation on the portable terminal apparatus. The portableterminal apparatus stores therein the input tags and time informationassociated with each other in any storage device. The respective tagsstored in the storage device and the taken image data of the videocamera are associated with each other, thereby enabling to obtaintag-given taken image data. In this case, it is assumed that the timeinformation of the video camera and the time information of the portableterminal apparatus are synchronized with each other.

Every time the tag is input, the portable terminal apparatus maytransmit the input contents to the video camera. When a communicationtime is negligibly small, the tag-given taken image data can beobtained.

Also in the above constitution, the portable terminal apparatus canautomatically identify the interface data in accordance with the imagingsituation and display the interface image generated based on theidentified interface data on the liquid crystal display. In other words,the interface data is automatically identified in accordance with theimaging situation, and the user can save the trouble of selecting theinterface.

Also in the application system, any processing of the central controller400 within the communication apparatus (the portable terminal apparatusin the above example) can be achieved by causing a central processingunit (CPU) to execute a computer program.

In a system in which the imaging apparatus or the portable terminalapparatus performs network communication including wirelesscommunication with an external server to transmit and receive variouskinds of data therebetween, the present invention can also be applied tothe external server.

Specifically, the external server acquires information on the imagingsituation such as the position information of the imaging apparatus orthe like by deriving it by itself, by receiving it from the imagingapparatus, or the like, and based on the information on the acquiredimaging situation, identifies the interface data appropriate for theimaging situation from among pieces of interface data stored in theserver. The identified interface data or interface image data generatedfrom the identified interface data is transmitted and output to theimaging apparatus or the like via network communication.

Thus, while reducing various kinds of processing loads of the imagingapparatus or the like, the user can use the interface image forinputting tags to the taken image data automatically changed inaccordance with the imaging situation.

The image processing apparatus, the method for processing an image, andthe computer program according to the present invention canautomatically change the interface image for inputting tags to the takenimage data in accordance with the imaging situation.

Although the invention has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. An image processing apparatus, comprising: ataken image data acquiring unit configured to acquire taken image data;an imaging situation information acquiring unit configured to acquire aposition information of a location for acquiring the taken image data asinformation on an imaging situation of the taken image data; a storageunit configured to store therein a plurality of pieces of interface datarelated to a tag to be attached to the taken image data and a historyinformation in which past position information of the location foracquiring the taken image data and a user interface which has been usedin the location for acquiring the taken image data and which can attachthe tag to the taken image data are associated with each other; aninterface identifying unit configured to identify a plurality of piecesof the interface data from among the pieces of the interface data storedin the storage unit based on current position information acquired bythe imaging situation information acquiring unit as well as the pastposition information of the location for acquiring the taken image dataand the user interface which has been used in the location for acquiringthe taken image data and associated to the past position information,both of them being included in the history information stored in thestorage unit; and a display image generating unit configured to generateinterface image data related to the user interface the interface dataidentified by the interface identifying unit and to generate image datafor display based on the generated interface image data and the takenimage data.
 2. The image processing apparatus according to claim 1,wherein the storage unit is further configured to store therein mapinformation indicating a correspondence relation between facilityinformation and facility position information, and the interfaceidentifying unit is further configured to identify the interface databased on the map information.
 3. The image processing apparatusaccording to claim 1, wherein the storage unit is further configured tostore therein person correspondence information indicating acorrespondence relation between the interface data and a person or agroup of persons as a subject to be imaged, the imaging situationinformation acquiring unit is configured to acquire subject informationon the subject to be imaged or the group of subjects to be imaged, andthe interface identifying unit is configured to identify the interfacedata based on the subject information and the person correspondenceinformation.
 4. The image processing apparatus according to claim 1,wherein the imaging situation information acquiring unit is configuredto generate image analysis information including information on at leastone of an amount of movement of an object, a voice, and presence orabsence of a certain subject to be imaged within the taken image data,and the interface identifying unit is configured to identify theinterface data based on the image analysis information.
 5. A method forprocessing an image to be executed in an image processing apparatus, themethod comprising: acquiring taken image data; acquiring a positioninformation of a location for acquiring the taken image data asinformation on an imaging situation of the taken image data; referringto a storage unit that stores therein a plurality of pieces of interfacedata related to a tag to be attached to the taken image data, andidentifying a plurality of pieces of the interface data from among thepieces of the interface data stored based on current positioninformation acquired and a history information in which past positioninformation of the location for acquiring the taken image data, a dateand a time of acquiring the taken image data, and a user interface whichhas been used in the location for acquiring the taken image data andwhich can attach the tag to the taken image data are associated witheach other; and generating interface image data related to the userinterface based on the identified interface data and generating imagedata for display based on the generated interface image data and thetaken image data.
 6. A non-transitory computer-readable storage mediumhaving stored thereon a computer program that causes a computer toexecute: acquiring taken image data; acquiring a position information ofa location for acquiring the taken image data as information on animaging situation of the taken image data; referring to a storage unitthat stores therein a plurality of pieces of interface data related to atag to be attached to the taken image data, and identifying a pluralityof pieces of the interface data from among the pieces of the interfacedata stored based on current position information acquired and a historyinformation in which past position information of the location foracquiring the taken image data, a date and a time of acquiring the takenimage data, and a user interface which has been used in the location foracquiring the taken image data and which can attach the tag to the takenimage data are associated with each other; and generating interfaceimage data related to the user interface based on the identifiedinterface data and generating image data for display based on thegenerated interface image data and the taken image data.
 7. An imageprocessing apparatus, comprising: an imaging situation informationacquiring unit configured to acquire a position information of alocation for acquiring the taken image data as information on an imagingsituation of the taken image data; a storage unit configured to storetherein a plurality of pieces of interface data related to a tag to beattached to taken image data and a history information in which pastposition information of the location of acquiring the taken image data,a date and a time of acquiring the taken image data and a user interfacewhich has been used in the location for acquiring the taken image dataand which can attach the tag to the taken image data are associated witheach other; an interface identifying unit configured to identify aplurality of pieces of the interface data from among the pieces of theinterface data stored in the storage unit based on current positioninformation acquired by the imaging situation information acquiring unitas well as the past position information of the location for acquiringthe taken image data and the user interface which has been used in thelocation for acquiring the taken image data and associated to the pastposition information, both of them being included in the historyinformation stored in the storage unit; and an output unit configured tooutput the interface data identified by the interface identifying unit.